Intent-aware keyboard

ABSTRACT

Systems, methods, and computer storage media having computer-executable instructions embodied thereon that provide contextual services are provided. Embodiments of the present invention allow an input mechanism to provide contextual services. Exemplary input mechanisms include a keyboard, a gesture interface, and a speech interface. These inputs may be used to provide user input into one or more applications running on a computer. The contextual services provided include composition assistance, grammatical assistance, communication-context assistance, and research assistance. In one embodiment, an input mechanism (“IME”) program provides the contextual service. The IME program may work with a remote contextual-service provider. The IME program communicates user input and contextual information to the contextual-service provider. The contextual-service provider analyzes the input and contextual information to determine whether one or more contextual services should be provided.

BACKGROUND

Many computing device users are habitual, or at least engage in somereasonably predictable behaviors on a routine basis. For instance, aparticular user may routinely conduct a search for “world news” firstthing in the morning to view headlines from prominent overnight newsstories. Or, a user may have been engaged in an email conversation witha friend early in the day and, a few hours later, conduct a search forinformation about a book the friend recommended during the conversation.While each of these scenarios is somewhat predictable, at least with adegree of confidence greater than that of purely random search behavior,they still require that the user access an online search application andtype one or more search query terms to access the information (and/orlinks to the information) she desires to access.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Embodiments of the present invention allow an input mechanism to providecontextual services. Exemplary input mechanisms include a keyboard, agesture interface, and a speech interface. These inputs may be used toprovide user input into one or more applications running on a computer.The user input may be a control input, such as selecting a button, orlanguage input, such as typing letters or characters. The contextualservices may be provided in conjunction with an interface into which thepresently received input is entered. For example, a contextual servicemay be provided to complement an email interface to which the user isadding text. The context of the active interface is used to determinethe content of the contextual service and which contextual service toprovide, if any.

The contextual services provided to the user include compositionassistance, grammatical assistance, communication-context assistance,research assistance, and disambiguation assistance. The compositionassistance helps a user compose textual content and may take the form ofan auto complete or auto suggest feature. The research assistanceprovides information or links to sources of information that may berelevant to an active interface. The research assistance may take theform of a contextual search result, a contextual query suggestion, andknowledgebase entries. Knowledgebase entries provide direct factualinformation about an object (e.g., a person, place, or activity) relatedto a subject matter present in an active interface. For example, thefactual information could include the location and elevation of amountain mentioned in an email being composed. The contextual searchresults may include a link to a webpage, but could also include asummary taken from a feed or other source providing information aboutthe news, weather, finance, sports scores, or the like. The searchresult could include images, audio files, videos, or applications thatmay help the user perform a task. The grammatical assistance may provideentries from a dictionary or usage guide. The communication-contextassistance may be search results showing social posts or communications(e.g. texts and emails) associated with an intended recipient of acommunication being drafted. For example, the communication assistancemay be links to social posts related to a recipient of an email beingdrafted. In another example, emails previously sent with similar subjectmatter may be provided as a search result. The disambiguation suggestionreviews content and present input to determine what the user isattempting to accomplish and then makes a suggestion consistent withaccomplishing the task.

A disambiguation assistance determines what a user input means in viewof context. (User intent is explained in more detail within the detaileddescription.) The disambiguation assistance may be provided directly toa computing application adapted to consume the information or provideddirectly to the user. The disambiguation assistance may help avoid asituation where a computer misunderstands a user's instruction for lackof context. For example, the user may enter a query resembling a commonrequest, such as “score of Spartan's game.” There are several well-knownschools or teams with the Spartan as a mascot, but the disambiguationservice may provide contextual information to clarify which Spartan teamor sport is intended. For example, the user may be writing an articleabout women's volleyball. The disambiguation service could provide acontextual interface which asks “Did you mean ‘Scores for the Spartan'swomen's volleyball.’” The result of intent disambiguation could beprovided directly and/or used to provide other contextual services. Forexample, search results for the women's volleyball team could beprovided.

In one embodiment, an input mechanism (“IME”) program provides thecontextual service. The IME program may work with a remotecontextual-service provider. The remote contextual-service provider mayoperate in a data center and provide contextual services to numerousdevices. The IME program communicates user input and contextualinformation to the contextual-service provider. The contextual-serviceprovider analyzes the input and contextual information to determinewhether one or more contextual services should be provided. Eachcontextual service may have a separate set of trigger mechanisms thatare used to determine when the contextual service would be useful to auser.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and not limitedin the accompanying figures in which like reference numerals indicatesimilar elements and in which:

FIG. 1 is a block diagram of an exemplary computing environment suitablefor implementing embodiments of the invention;

FIG. 2 is a diagram of a computing environment suitable for providingcontextual-services, in accordance with an embodiment of the presentinvention;

FIG. 3 is a diagram of a word processor's user interface presentingcomposition assistance, knowledge base info, and a contextual search, inaccordance with an embodiment of the present invention;

FIG. 4 is a sequence diagram illustrating communications occurringbetween a computing device and a remote contextual-service component asa contextual service is provided to the computing device by the remotecontextual-service component, in accordance with an embodiment of thepresent invention;

FIG. 5 is a diagram of a word processor's user interface presenting acontextual search contemporaneously with query input box, in accordancewith an embodiment of the present invention;

FIG. 6 is a flow diagram showing a method for providing contextualservices in association with an input mechanism, in accordance with anembodiment of the present invention;

FIG. 7 is a flow diagram showing a method for providing contextualservices, in accordance with an embodiment of the present invention; and

FIG. 8 is a flow diagram showing a method for providing a contextualservice interface, in accordance with an embodiment of the presentinvention.

DETAILED DESCRIPTION

The subject matter of the present invention is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

Embodiments of the present invention allow an input mechanism to providecontextual services. Exemplary input mechanisms include a keyboard, agesture interface, and a speech interface. These inputs may be used toprovide user input into one or more applications running on a computer.The user input may be a control input, such as selecting a button, orlanguage input, such as typing letters or characters. The contextualservices may be provided in conjunction with an interface into which thepresently received input is entered. For example, a contextual servicemay be provided to complement an email interface to which the user isadding text. The context of the active interface is used to determinethe content of the contextual service and which contextual service toprovide, if any.

The contextual services provided include composition assistance,grammatical assistance, communication-context assistance, researchassistance, and disambiguation assistance. The composition assistancehelps a user compose textual content and may take the form of an autocomplete or auto suggest feature. The research assistance providesinformation or links to sources of information that may be relevant toan active interface. The research assistance may take the form of acontextual search result, a contextual query suggestion, andknowledgebase entries. Knowledgebase entries provide direct factualinformation about an object (e.g., a person, place, or activity) relatedto a subject matter present in an active interface. For example, thefactual information could include the elevation of a mountain mentionedin an active interface. The contextual search results may include a linkto a webpage, but could also include a summary taken from a feed orother source providing information about the news, weather, finance,sports scores, or the like. The search result could include images,audio files, videos, or applications that may help the user perform atask. The grammatical assistance may provide entries from a dictionaryor usage guide. The communication-context assistance may be searchresults showing social posts or communications (e.g. texts and emails)associated with an intended recipient of a communication being drafted.For example, the communication assistance may be links to social postsrelated to a recipient of an email being drafted. In another example,emails previously sent with similar subject matter may be provided as asearch result.

A disambiguation assistance determines what a user input means in viewof context. (User intent is explained in more detail within the detaileddescription.) The disambiguation assistance may be provided directly toa computing application adapted to consume the information or provideddirectly to the user. The disambiguation assistance may help avoid asituation where a computer misunderstands a user's instruction for lackof context. For example, the user may enter a query resembling a commonrequest, such as “score of Spartan's game.” There are several well-knownschools or teams with the Spartan as a mascot, but the disambiguationservice may provide contextual information to clarify which Spartan teamor sport is intended. For example, the user may be writing an articleabout women's volleyball. The disambiguation service could provide acontextual interface which asks “Did you mean ‘Scores for the Spartan'swomen's volleyball.’” The result of intent disambiguation could beprovided directly and/or used to provide other contextual services. Forexample, search results for the women's volleyball team could beprovided.

In one embodiment, an input mechanism (“IME”) program provides thecontextual service. The IME program enables operation of the inputmechanism. In one embodiment, the IME program is part of an operatingsystem miming on a computing device. In another embodiment, the IME isan application that translates input data received from the inputmechanism hardware into input that is consumable by multipleapplications. For example, the IME may provide a touch screen keyboardthat is usable with multiple applications on a mobile device.

The IME program may work with a remote contextual-service provider. Theremote contextual-service provider may operate in a data center andprovide contextual services to numerous devices. The IME programcommunicates user input and contextual information to thecontextual-service provider. The contextual-service provider analyzesthe input and contextual information to determine whether one or morecontextual services should be provided. Each contextual service may havea separate set of trigger mechanisms that are used to determine when thecontextual service would be useful to a user.

The IME program may communicate contextual information to thecontextual-service provider, in addition to input (e.g. keystrokes).Contextual information associated with a user is identified and utilizedto predict one or more contextual services that the user is likely todesire to access at a particular point in time, e.g., the point in timeat which the contextual services are determined. Contextual informationmay describe habitual or routine behaviors of the user and/or indicatorsassociated with events, activities or behaviors of the user and mayinclude, by way of example only, routine search behaviors, recent textand/or email conversations engaged in by the user, events scheduled onthe user's electronic calendar, multimedia events engaged in by the user(such as listening to music, watching television programming, etc.), andthe like. For instance, a user may routinely search for traffic reportsaround 7:00 a.m. Thus, contextual services provided during the earlymorning hours may include traffic information (and/or links thereto).Contextual information may include a user's present location. Thelocation may be determined by a location aware device, such as a smartphone, using GPS or other location technology.

Or, a user's electronic calendar may indicate they have a lunchappointment across town. In this instance, contextual services providedduring the two hours prior to the lunch appointment may includedirections to the restaurant and/or a link to the restaurant's lunchspecials for the day. The identified contextual services are contextualservices that, based on the context surrounding the user eitherroutinely/habitually or specific to a particular point in time, the userhas an increased likelihood (relative to information unrelated to thecurrent context surrounding the user) of desiring to access.

Upon receiving an indication that the user desires to perform acontext-aware search, the identified contextual services (or linksthereto) are presented automatically to the user, that is, without theuser having to input any search query terms. The indication that theuser desires to perform a context-aware search may be received, forinstance, upon receiving an indication that has paused while composing adocument. As the identified contextual services are presented withoutuser input of any search query terms into the contextual search, thecontextual services are quickly and efficiently presented to the user,with minimal user effort.

Also presented within a contextual service box is a search query inputregion configured for receiving user-input search query terms. As such,if the presented information (and/or information links) does not alignwith what the user wanted to have presented upon selecting thecontext-aware search button, he or she may input search query terms anda search will be conducted for information related to the input terms ina conventional manner, though with an enhanced assessment of the intentof the user which may prove beneficial in determining search results.

A user's response to an offered contextual service may be tracked andused to refine the system used to determine whether a particularcontextual service is offered given a set of contextual circumstances.The actions of many users may be tracked and recorded on a per userbasis, aggregated into groups of users with similar characteristics orprofiles, or aggregated across all users to determine when to offer agiven service. The characteristics may be aggregated by othercharacteristics, for example time of day, active application or task.For example, user of contextual services and other activities ofinterest conducted during the evening may be aggregated together. Inanother example, actions performed while composing a document within aword processing application may be aggregated together. The response tocontextual services in combination with other contextual data, useractions and the like may be analyzed to determine use patterns.

The patterns can be analyzed to ascertain a probability that a user willtake an subsequent action given a series of past actions (e.g.,inputting a work or phrase) and a present context (e.g., contents andcharacteristics of an active interface). When the probability exceeds aconfidence threshold, then the user may be said to intend to perform thesubsequent action. As used herein the expression “user intent.”“intent,” or similar means a statistical likelihood of performing anaction, rather than a subjective state of mind. Any given user may nothave a specific intent, but in the aggregate, a statisticallysignificant portion of similarly situated user's will be likely to havean intent as measured by performing one or more actions mapped to theintent. For example, use patterns may be analyzed to determine that in agiven context (e.g., requesting directions to an out of towndestination), a significant portion of users may perform a search forweather. In this example, a user performing a requesting out of towndirections may be said to have a search intent for weather informationat the destination.

Some of the contextual services may be implemented on a client device(e.g., a smartphone, tablet, PC, and the like) directly connected withthe IME without real-time input from the remote contextual-serviceprovider. The contextual-service provider may provide content used toprovide contextual services that are stored on the client until a needis triggers. In other embodiments, the contextual-service is providedonly using real-time input from the remote contextual-service provider.

The input mechanism may have a button or other control to turn thecontextual function on and off. For example, a keyboard may provide acontextual service button. When the contextual service is on, thecontextual services are periodically provided when circumstancesindicate that a user is likely to appreciate and use the service. Alogin to the contextual service may be provided to the user. Once a userexplicitly logs into a service a cookie or other identificationmechanism may be used to associate the device with the user. Logginginto the service allows several of the user's devices to be associatedwith the user's contextual service account. Multi-user devices, such asPCs and gaming consoles, may associate different active profiles on themulti-user device with different users. In one embodiment, the user doesnot need to log into the contextual service explicitly. Logins to otherservices can be used to associate multiple users with the samecontextual services profile. For example, a user logging into a searchservice, a social network, online email, online productivityapplications, and the like from multiple devices may allow the multipledevices to be associated with a single user profile.

Having briefly described an overview of embodiments of the invention, anexemplary operating environment suitable for use in implementingembodiments of the invention is described below.

Exemplary Operating Environment

Referring to the drawings in general, and initially to FIG. 1 inparticular, an exemplary operating environment for implementingembodiments of the invention is shown and designated generally ascomputing device 100. Computing device 100 is but one example of asuitable computing environment and is not intended to suggest anylimitation as to the scope of use or functionality of the invention.Neither should the computing device 100 be interpreted as having anydependency or requirement relating to any one or combination ofcomponents illustrated.

The invention may be described in the general context of computer codeor machine-useable instructions, including computer-executableinstructions such as program components, being executed by a computer orother machine, such as a personal data assistant or other handhelddevice. Generally, program components, including routines, programs,objects, components, data structures, and the like, refer to code thatperforms particular tasks or implements particular abstract data types.Embodiments of the invention may be practiced in a variety of systemconfigurations, including handheld devices, consumer electronics,general-purpose computers, specialty computing devices, etc. Embodimentsof the invention may also be practiced in distributed computingenvironments where tasks are performed by remote-processing devices thatare linked through a communications network.

With continued reference to FIG. 1, computing device 100 includes a bus110 that directly or indirectly couples the following devices: memory112, one or more processors 114, one or more presentation components116, input/output (I/O) ports 118, I/O components 120, and anillustrative power supply 122. Bus 110 represents what may be one ormore busses (such as an address bus, data bus, or combination thereof).Although the various blocks of FIG. 1 are shown with lines for the sakeof clarity, in reality, delineating various components is not so clear,and metaphorically, the lines would more accurately be grey and fuzzy.For example, one may consider a presentation component such as a displaydevice to be an I/O component 120. Also, processors have memory. Theinventors hereof recognize that such is the nature of the art, andreiterate that the diagram of FIG. 1 is merely illustrative of anexemplary computing device that can be used in connection with one ormore embodiments of the invention. Distinction is not made between suchcategories as “workstation,” “server,” “laptop,” “handheld device,”etc., as all are contemplated within the scope of FIG. 1 and refer to“computer” or “computing device.”

Computing device 100 typically includes a variety of computer-readablemedia. Computer-readable media can be any available media that can beaccessed by computing device 100 and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable media may comprise computerstorage media and communication media. Computer storage media includesboth volatile and nonvolatile, removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer-readable instructions, data structures, program modules orother data.

Computer storage media includes RAM, ROM, EEPROM, flash memory or othermemory technology, CD-ROM, digital versatile disks (DVD) or otheroptical disk storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices. Computer storage media doesnot comprise a propagated data signal.

Communication media typically embodies computer-readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared and other wireless media. Communication mediadoes not fall within the scope of computer-storage media. Combinationsof any of the above should also be included within the scope ofcomputer-readable media.

Memory 112 includes computer-storage media in the form of volatileand/or nonvolatile memory. The memory 112 may be removable,nonremovable, or a combination thereof. Exemplary memory includessolid-state memory, hard drives, optical-disc drives, etc. Computingdevice 100 includes one or more processors 114 that read data fromvarious entities such as bus 110, memory 112 or I/O components 120.Presentation component(s) 116 present data indications to a user orother device. Exemplary presentation components 116 include a displaydevice, speaker, printing component, vibrating component, etc. I/O ports118 allow computing device 100 to be logically coupled to other devicesincluding I/O components 120, some of which may be built in.Illustrative I/O components 120 include a microphone, joystick, gamepad, satellite dish, scanner, printer, wireless device, etc.

Exemplary Online Gaming Environment

Turning now to FIG. 2, an exemplary computing system environment 200 forproviding contextual services in association with a keyboard, or otherIME, is shown, in accordance with an embodiment of the presentinvention. The computing system environment 200 shown in FIG. 2 ismerely an example of one suitable computing system environment and isnot intended to suggest any limitation as to the scope of use orfunctionality of embodiments of the present invention. Neither shouldthe computing system environment 200 be interpreted as having anydependency or requirement related to any single module/component orcombination of modules/components illustrated therein.

The components/modules illustrated in FIG. 2 are exemplary in nature andin number and should not be construed as limiting. Any number ofcomponents/modules may be employed to achieve the desired functionalitywithin the scope of embodiments hereof. Further, some of thecomponents/modules may be located on any number of servers or clientcomputing devices. By way of example only, the contextual-servicecomponent 240 might reside on a server, cluster of servers, or acomputing device remote from one or more of the remaining components.

It should be understood that this and other arrangements describedherein are set forth only as examples. Other arrangements and elements(e.g., machines, interfaces, functions, orders, and groupings offunctions, etc.) can be used in addition to or instead of those shown,and some elements may be omitted altogether. Further, many of theelements described herein are functional entities that may beimplemented as discrete or distributed components or in conjunction withother components/modules, and in any suitable combination and location.Various functions described herein as being performed by one or moreentities may be carried out by hardware, firmware, and/or software. Forinstance, various functions may be carried out by a processor executinginstructions stored in memory.

FIG. 2 will be described with reference to using a keyboard as the inputmechanism “IME,” but embodiments the present invention are not limitedfor use with a keyboard. The IME could be speech recognition technology,touch screen, or some other input mechanism. The computing systemenvironment 200 includes device 220 and device 221 communicativelycoupled to productivity server 260, gaming server 270, and contextualservices component 240 from network 210. Network 210 may include,without limitation, one or more local area networks (LANs) and/or widearea networks (WANs). Such networking environments are commonplace inoffices, enterprise-wide computer networks, intranets and the Internet.Accordingly, the network 116 is not further described herein.

Computing devices 220 and 221 may be smart phones, slates, e-readers,personal computers, or the like. The devices 220 and 221 may havecomponents similar to those described with reference to computing device100. The devices 220 and 221 have a display device or are coupled to anindependent display device that is configured to display information tothe user, for instance, information relevant to communications initiatedby and/or received by the devices 220 and 221, browsing activity,contextual services, and the like. Embodiments are not intended to belimited to visual display but rather may also include audiopresentation, combined audio/visual presentation, and the like.

Device 220 includes an IME program 222, an IME data store 224, and agroup of applications 225. The group of applications 225 could includehundreds or thousands of applications including a word processor, webbrowser, spreadsheet, database, game, book application, media player,and other applications. In addition, device 220 may have an operatingsystem. In one embodiment, the IME program 222 is part of the operatingsystem. In another embodiment, the IME program 222 is separate from theoperating system but closely associated with the input mechanism (e.g.,keyboard).

The IME program 222 may be associated with multiple input mechanisms.For example, the IME program 222 may provide contextual capabilities toa hardware keyboard, a virtual or touch-screen keyboard, and speechrecognition technology. Alternatively, each input mechanism may have itsown dedicated IME program. Only one IME program 222 is shown for thesake of simplicity in FIG. 2.

The IME data store 224 saves contextual information relevant to device220. In various embodiments, such information may include, user profiledata, computing device state data, routine search behaviors, recent textand/or email conversations engaged in by the user, events scheduled onthe user's electronic calendar, multimedia events engaged in by the user(such as listening to music, playing a game, watching televisionprogramming, etc.), links to uniform resource locators (URLs) frequentedby the user, links to URLs having information associated with eventsscheduled on the user's electronic calendar and/or with keywordsextracted from text and/or email conversations engaged in by the user,and/or the like. The profile data may include writing or speakingpatterns for a unique user. In embodiments, the IME data store 224 isconfigured to be searchable for one or more of the items stored inassociation therewith. Though illustrated as a single, independentcomponent, the data store 224 may, in fact, be a plurality of storagedevices, for instance a database cluster.

In addition to contextual data specific to the device 220 or multipledevices associated with a user, the IME data store 224 may storeinformation used to provide a contextual service. For example, userpreferences for how and when contextual services are provided may berecorded in the IME data store 224. The user preferences may include anopt-in or opt-out to policies that determine how contextual informationis used or shared. The data in the IME data store 224 may becommunicated to the IME interface 242 and combined with informationreceived from a user's other devices in the user's contextual serviceprofile. As mentioned, a login or other method may be used to associatemultiple devices with a single user. As explained subsequently, theintent component 248 may use contextual information from several devicesto provide a service or determine which service to provide.

The device 221 also includes IME program 222. This illustrates that thesame IME functionality may reside on multiple computing devices. IMEdata store 226 may have a similar function and type of information aspreviously described with IME data store 224. However, some of theinformation within IME data store 226 may be different from that in IMEdata store 224 because it relates to activities occurring on theparticular device. The IME data store 226 may also differentiate betweenmultiple users known to use device 221. Device 221 includes a group ofapplications 227.

The productivity server 260 includes software and hardware to enable aremote productivity service. The remote productivity surfaces may bedescribed as a cloud based productivity solution. For example, theproductivity server 260 may enable device 220 to access a remote wordprocessing application, remote e-mail application, remote databaseapplication, remote video editing application, and the like.

The gaming server 270 provides a remote gaming experience for clientdevices. The games may be rendered by gaming server 270 and manipulatedin response to inputs received from client devices. In anotherembodiment, the gaming server 270 enables multiplayer games by trackinginteractions between players and the gaming environment remotely whilethe game images are generated by the client devices. The devices 220 and221 could be game consoles.

The contextual services component 240 provides contextual services todevice 220 in cooperation with IME program 222. The contextual servicescomponent 240 may operate in a data center or other remote computingfacility that is capable of interacting with numerous userssimultaneously. The contextual services component 240 comprises an IMEinterface 242, a composition component 244, a search component 246, aselection component 248, an auto suggest component 250, aknowledge-based component 252, and the social component 254.

The IME interface 242 receives contextual information from IME program222 and distributes the information to other components within thecontextual service component 240. The IME interface 242 also returnscontextual services that are relevant to the contextual information. Thecontextual services are communicated to the IME program 222, which inturn takes steps to output the contextual service to the user.

The composition component 244 uses contextual information to help theuser compose text. The compositional service may be provided as an autosuggest word or auto complete word. The auto complete word attempts todetermine what word or words the user is attempting to input based on atext fragment received as part of the contextual information. The autocomplete suggestion starts with the text fragment and adds characters toform a word or phrase. The composition component 244 may use adictionary that returns commonly used phrases that are consistent withthe text fragment. The composition component 244 may also take intoaccount words used within an active document or e-mail to determinewhich word the user is mostly likely to be entering. When the activeapplication is an e-mail application, words within an e-mailconversation related to an email being composed may be evaluated toprioritize which words the user is likely entering. Similarly, theuser's vocabulary may be analyzed based on other documents generated bythe user to determine which word the user is entering.

The search component 246 returns search results that are responsive to atext fragment or recently entered text. In addition to returning searchresults, suggested search queries may be returned by the searchcomponent 246. The suggested search queries and common search result maybe based on searches conducted by numerous users to determine the likelysearch query. The search result could include images, audio files,videos, or applications that may help the user perform a task. Thesearch results could include links to webpages and advertisements.

The search component 246 may return posts from the user's socialnetwork. For example, a social post from a recipient of an email beingdrafted may be presented when keywords within the email match keywordswithin the social post. In an example, a user is currently attending aball game at a particular arena. The search component 246 utilizes thestate information (location and time) of a device to automaticallyprovide output of a game roster and player statistics.

In another example, a user visits a large retail store and finds an itemhe may wish to purchase. He utilizes the computing device camera, whichis another example of an IME program, (e.g., cell phone camera) to takea picture of the barcode. On the computing, the barcode is sent to anapplication that reads the barcode and provides information. At the sametime, the barcode is sent to search component 246 and a price comparisonpage for the item is provided as a contextual service. Still further, auser checks into a hotel in city X, opens a browser, and navigates tosearch engine. As the system is aware this is the time the usergenerally eats dinner, a listing of nearby restaurants for the user'sfavorite cuisine, as well as a map and a mobile coupon are presented allas part of the contextual service. The user did not even need to enter aquery, but an intent to search and the purpose of the search wasdetermined by context.

The selection component 248 determines which contextual service isprovided to the user at a point in time, if any by evaluating thecontextual information to determine a user's active task and likely needfor a contextual service. Upon making an intent determination, aninstruction is sent to one or more other components to generate theservice. In another embodiment, the various components present orgenerate their respective contextual services but the selectioncomponent 248 determines whether or not the service should be providedto the user at the present time. For example, the selection component248 may determine that the user has not demonstrated an intent tosearch, and therefore, search results or suggested search queries, arenot presented to the user. Instead, the composition component 242provides auto completion suggestions that are presented to the user inresponses to an intent to compose.

The selection component 248 is configured for selecting at least onecontextual service using the contextual information received by the IMEinterface 242. The selection component 248 may comprise one or morerules that utilize the contextual information to determine contextualservices that the user has an increased likelihood of using. Theselection component 248 may include utilization of profile dataassociated with the user, data associated with the current state of thecomputing device (which may include a current time and/or location ofthe computing device), or a combination of profile data and currentstate data to select the at least one contextual service. Inembodiments, profile data may be provided directly by the user and mayinclude information such as age, sex, marital status, home and workaddresses, or the like. Profile data may also include preference data,such as a particular coffee shop, fast food franchises, and so forthpreferred by the user. Dislikes may also be provided. The profileinformation may include a history or the user's interactions with thecontextual services. The profile information may include a browsinghistory, a search history, and history of social interactions, includingexpressions of affirmation or disapproval for a business, individual, ororganization. In embodiments, the at least one contextual serviceincludes one of visible information, audible information, or acombination of visible and audible information.

The selection component 248 may mine data associated with the userand/or the computing device for use in selecting appropriate contextualservices for output. For instance, the selection component 248 may beconfigured to mine at least a portion of profile data associated with auser, data associated with a current state of the computing device, dataindicative of a user's interactions with the device, user-activity data,data cached in association with the computing device and/or thecontextual services used previously, and/or any combination thereof.Data may be mined from the user's activities and other data sourcesoccasionally, e.g., periodically and/or after some threshold amount ofactivity is reached, such as every fifty telephone calls. Examples ofdata mined from such activities include telephone numbers frequentlycalled, calling patterns (e.g., analyzed through collaborative filteringand graph analysis), email addresses to which messages are regularlysent, other searches and advertisements, other application logs,location tracks, SMS content, and so forth. Examples of otherinformation include things like social networking buddy lists, pastpurchases made via this device or an associated device, and the like.Browsing history, search history, interactions within a social network,and responses to contextual services may also be gathered. Inembodiments, various profile data may be merged into a suitable datastructure and, if desired, persisted in association with the user ordevice, such as by the device's telephone number. In this manner,profile data is regularly available to assist in decision making withrespect to selecting contextual services to present, as more fullydescribed below.

In embodiments, as part of the contextual service selection, keywordsare extracted from data that is mined or otherwise received (e.g., byreceiving IME interface 242) and such extracted keywords are utilized asinput into a search engine to aid in determining appropriate contextualservices to provide for output. For instance, if a user receives an SMSthat reads “let's meet at restaurant X at 7:00 pm,” the keyword“restaurant X” may be extracted and utilized as input into a searchengine to cause retrieval of a map to and menu for restaurant X (or alink thereto). Subsequently, upon receiving an indication that the userdesires a context-aware search to be conducted, the map and menu orlinks thereto may be automatically presented—without the user inputtingany keywords into the search engine themselves. Such contextual serviceretrieval may take place because the user is viewing the SMS message atthe time the context-aware search is requested and/or because acontext-aware search is requested in close time proximity to the 7:00 pmdinner appointment. In embodiments, the contextual service selectionlogic utilizes likelihoods/probabilities to reconcile which of multiplecontextual services to provide for presentation to the user. Each ofthese embodiments is more fully described below.

The selection component 248 is configured for determining which of aplurality of contextual indicators are more likely to generate desiredcontextual services and/or which of a plurality of contextual servicesare most likely to be desirable to the user to have presented at aparticular point in time. For instance, if a user's electronic calendarindicates s/he is to meet a friend for dinner at 7:00 pm at a particularrestaurant, is receipt of a contextual indicator of the friend'sidentity, the current location of the user, and/or the identity of therestaurant more likely to generate contextual services the user desiresto view at a particular point in time than a contextual indicator thatthe user routinely checks traffic conditions at 6:00 pm before leavinghis or her office? Or, is a contextual service linking to the friend'ssocial networking page, a contextual service indicative of therestaurant menu, a contextual service linking to directions from theuser's current location to the restaurant, and/or a contextual servicelinking to traffic conditions between the user's current location andthe restaurant more likely to be desirable to the user to view?

In an embodiment, the selection component 248 is configured to receiveinput of a plurality of contextual services and utilize one or morestatistical methodologies to assign a probability (or likelihood) toeach. Those contextual indicators and/or contextual services having thehighest probability or likelihood that the user desires to view them,when taken into context of the remaining contextual indicators and/orcontextual services, are utilized to provide the appropriate output tothe user, as more fully described below. Embodiments hereof are notlimited to uses of any particular statistical methodology and,accordingly, particular methodologies are not further described herein.

The auto suggest component 250 suggests various tasks or points outcapabilities to the user that may help complete a task the user has ademonstrated (implicitly or explicitly) a present intent to complete.For example, a user attempting to edit a picture may be provided a linkto the productivity service 260 that allows the user to complete thepicture editing using the productivity service 260s picture editingservice. In another embodiment, if the user appears to be shopping, orresearching a product, specific websites or other resources that areable to help the user with that task are suggested.

The knowledge-based component 252 provides factual content related tothe user's recent input. For example, when the user types in “George B,”the knowledge base may provide information about President George H. W.Bush and/or President George W. Bush. The context may be considered, forexample, if the user is writing about the war on terror, then thefactual information may relate only to George W. Bush the 43^(rd)president. Similarly, if the user's recent input described Mount St.Helens, then factual information related to Mount St. Helens may bepresented by the knowledge-based component 252.

The social component 254 surfaces information related to their presentinput from their social network. For example, when the user is composingan e-mail, previous posts by a recipient of the e-mail may be summarizedand described. For example, a user browses her contact list until shearrives at John Doe's listing, at which point the context-aware searchfeature is provided. In this instance, John Doe's social networking pageis presented for review before making the call.

Turning now to FIG. 3, contextual interface services are illustrated, inaccordance with an embodiment of the present invention. FIG. 3 shows anapplication window 300 for a word processing application. Embodiments ofthe present invention are not limited to operation with a wordprocessing application. For example, the contextual interface servicescould be provided in an email application, spreadsheet, search engineinterface, and social media application. The application window 300includes text 312, the last sentence of which states, “we recommendusing Windows' encryp.” In this case, “encryp” is a text fragment 314.Composition-assistance interface 315 includes an autocomplete 316suggestion, which is the word “encryption.” The autocomplete serviceworks using a dictionary, or other corpus of words, to add letters tothe text fragment 314 to form a known word or phrase. The secondsuggestion “BITLOCKER®” 318 is not an autocomplete. Instead,“BITLOCKER®” 318 is an example of an autosuggest. The autosuggest is notbased on completing a word or phrase by adding letters to a textfragment, but instead evaluates the context of the text fragment tosuggest a word or phrase consistent with the meaning of what the user istyping. Here, BITLOCKER® is the name of Windows' encryption software.Selecting BITLOCKER® could replace “Windows' encrypt” or just the“encrypt” portion. The user may select the composition suggestioninstead of typing and the word and the selected word will be used toreplace the text fragment 314.

FIG. 3 also includes a research assistance interface 320. The content ofthe research assistance interface 320 provides additional informationabout subject matter described in the text 312. The additionalinformation could be in the form of search results, knowledge basedentries, suggested queries, and the like. In this case, the researchassistance interface 320 includes knowledge based entries resource A 322and resource B 324, both related to BITLOCKER®. In one embodiment, theresearch assistance interface 320 includes an input box (not shown) thatis automatically populated with user input as it is added to the text312. When the words form a query of interest, then user need only selecta search button (not shown) in the research assistance interface 320 torequest the search. Contextual information may be used in conjunctionwith the query to form search results that are presented in an updatedsearch result interface.

When a new word begins or a word is selected from the compositionassistance interface 315, then a new is automatically conducted. In oneembodiment, only keywords are considered for submission in query. Inanother embodiment, the user is able to type search terms into an inputbox associated with the research assistance interface 320. The search isconducted using contextual information from document 310.

Turning now to FIG. 4, a sequence diagram showing communicationsoccurring between components used to provide a contextual service isshown, in accordance with embodiment of the present invention. Thecomponents in FIG. 4 include: a keyboard 410; an IME component 412; aword processing application 414; and a contextual service component 416.

The keyboard 410 may be a separate piece of hardware or a soft-keyboardpresented on a touch screen. The IME component 412 enables thefunctionality of the keyboard, 410. The IME component 412 may beintegrated into or part of an operating system. The IME component 412enables the keyboard 410 to operate as an input device across multipleapplications.

The word-processing application 414 is an exemplary application that mayconsume the keyboard's 410 input. Embodiments of the present inventionalso work with applications such as databases, spreadsheets, emailapplications, games, entertainment services, web browsers, and the like.The application 414 may be based on a client device associated with thekeyboard or be provided by a server. For example, the word processingcomponent 414 could be provided by a remote service accessed by theclient device.

The contextual service component 416 may be similar to the contextualservice component 240, described previously with reference to FIG. 2.The contextual service component 416 provides contextual services tomultiple computing devices. The IME component 412 and the contextualservice component 416 may operate on different computing devices. Forexample, the contextual service component 416 may operate or be providedby a data center that connects with a computing device that isassociated with the keyboard 410, the IME component 412, and the wordprocessing application 414. As mentioned with reference to FIG. 2, thedata center serving the contextual service component 416 may communicatewith one or more computing devices over a wide area network, such as theInternet.

The IME component 412 communicates contextual information 418 to thecontextual service component 416. As mentioned, the contextualinformation 418 may be transmitted over a network such as the Internet.The contextual information 418 includes information about the computingdevice and the user or users of the computing device. Information aboutthe user may include account information or other information thatallows the user to be associated with a user profile accessible by thecontextual service component 416 across multiple devices, applications,and experiences. The contextual information 418 may include informationabout a user's preferences or habits. For example, the users browsinghistory, search history, and other activities performed on the computingdevice may be provided. The contextual information could includeinformation needed for the contextual service component 418 to accessother user accounts, for example social networks, remote storageservices, email accounts, and the like. In addition, demographicinformation about the user may be provided when available. The IMEcomponent 412 may provide user opt-ins and opt-outs to specify what typeof information may be provided to the contextual service component 416.An interface may be provided by the IME component 412 for this purpose.

The contextual information 418 also includes information about a presentactivity. For example, the contextual information 418 may includeinformation describing an open document in the active application. Forexample, when the active application is a word processing applicationthe contextual information could include key words or text within theopen word processing document. If the open application was an email ortexting application, key words from an open email chain, which couldinclude multiple communications between the same parties with a relatedsubject matter, could be provided. Contextual information helps thecontextual service component 416 understand what the user's presentintent is and what contextual services might be helpful to the user.

The keyboard 410 receives keystrokes from the user. These keystrokes 420are communicated to the IME component 412. The keystrokes 420 are alsocommunicated to the word processing application 414 by the IME component412. The word processing application 414 will manipulate the opendocument in response to the keystrokes. For example, if a user iscomposing a word, then the letters or numerals corresponding to thekeystrokes will appear on the screen as they are received by the wordprocessing application 414.

Groups of keystrokes or the individual keystrokes 420 are communicatedto the contextual service component 416 by the IME component 412. Thekeystrokes 420 are evaluated 422 by the contextual service component 416to determine whether a relevant contextual service may be appropriate.The types of contextual services that may be provided have beendescribed previously.

When the evaluation 422 indicates that a contextual service would helpthe user complete a task, such as composing a word or conductingresearch, the contextual service instruction 424 is communicated to theIME component 412. The IME component 412 then generates an interfacethat presents one or more contextual services to the user. As shown inFIG. 3, and as subsequently explained in FIG. 5, the interface may be anoverlay of an active interface. Upon receiving a request 438 from theuser through the keyboard 410, or through some other input mechanism,such as a mouse, voice command, or touch screen, the contextual serviceis provided. For example, the IME component may delete words or recentkeystrokes and replace them with a word selected by the user through thecontextual interface. Using the example of FIG. 3, the IME component 412may delete the letters in “encrypt” and replace them with the word orkeystrokes forming the word “encryption.” The communications shown inFIG. 4 may repeat as additional keystrokes are added and contextualservices are selected.

Turning now to FIG. 5, a contextual search service is illustrated inaccordance with an embodiment of the present invention. FIG. 5illustrates a web browser 500 showing a web page 505 related to apresidential library and museum. The webpage 505 includes a search box510. As can be seen, keystrokes 512 forming “George B” have been enteredinto the search box 510. The search box has a native auto-complete 514feature that automatically presents some query suggestions.

The IME component on the computing device sends the keystrokes 512 to acontextual service provider that provides instructions for a contextualsearch interface 520. The contextual search interface 520 may bepresented in an overlay that does not overlap the search box 510. Inother words, the contextual search interface 520 may be presented as analternative that the user can navigate to and select one or more searchresults or suggested queries if they are unsatisfied with the nativeauto-complete 514 feature associated with the search box 510.

The contextual search interface 520 includes three sections of usefulsearch information. The first section includes a knowledge base section521 that presents entries related to the intended search. The firstentry indicates that George H. W. Bush is the 41^(st) president 522. Theuser can learn more about the 41^(st) president by selecting the “more”button, 523. This could open a new interface or cause the user tonavigate to a portion of the website related to the 41^(st) president orto a separate website about the 41^(st) president. The second knowledgeentry indicates that George W. Bush is the 43^(rd) president 524. Theuser can learn more about the 43^(rd) president by selecting the “more”button 525.

The second section of the contextual search interface 520 is a searchresults section 530. The search results section 530 includes a firstsearch result 532 and a second search result 534. These two may be themost relevant search results returned in a response to a presumptivesearch query related to “George B.” For example, the search resultscould be related to George Bush. However, contextual information may beused to disambiguate between different George Bushes. For example, ifthe user was recently working on a document related to George Bush the43^(rd) president then both of the search results could be related toGeorge W. Bush the 43^(rd) president.

The search section 540 of the contextual search interface 520 includessuggested search queries. Selecting either “George Bush 41” 542 or“George Bush 43” 544 will cause search results to be presented in aseparate interface generated by the contextual search service. Inaddition, selecting a suggested search query, such as “George Bush 41”542, will cause the search query to be entered into search box 510. Thisgives this user the alternative to see results presented by the nativesearch interface on webpage 505 and search results presented by thecontextual service.

Turning now to FIG. 6, a method 600 for providing contextual services inassociation with an input mechanism is shown, in accordance with anembodiment of the present invention. The input mechanism may be a hardkeyboard, a soft keyboard, a speech-recognition mechanism, ahand-writing recognition system, or the like. Contextual servicesinclude composition assistance, search assistance, grammaticalassistance, and a knowledge base. The contextual service of method 600may be provided by a contextual-service provider that is remote from acomputing device associated with the input mechanism.

At step 610, an input fragment is received from an input mechanism(“IME”) program running on a computing device. The IME program may runon a client device that is remote from a contextual service providerthat receives the input fragment. The input fragment is less than a fullinput, for example letters forming less than a phrase or a command. Whatconstitutes an input fragment may vary according to the input mechanism.For example, the input fragment may be part of the letters in a word orphrase when a keyboard is the input mechanism. The input fragment couldbe part of a gesture or audio input. The input fragment may be receivedat a contextual service provider that is remote from the computingdevice. For example, a text fragment may be communicated over a networksuch as the Internet. The text fragment comprises letters that do notpresently constitute a full word. When evaluating whether a textfragment comprises a full word, the letters are evaluated in the orderin which they are received. The fact that the letters could berearranged to form a full word is not relevant to the analysis of step610.

At step 620, information is received from the IME program. Theinformation is received at the contextual-service provider. Theinformation describes a characteristic of an active application intowhich the IME program input the input fragment. For example,characteristics of a word processing document in an active wordprocessing application may be communicated. Similarly, the informationcould include characteristics of an email that is being edited in anactive email application.

At step 630, the characteristic and the input fragment are determined tosatisfy a contextual service's trigger criteria. Each contextual servicemay have a separate trigger criteria. The characteristic and the inputfragment may satisfy the contextual criteria of multiple contextualservices. The trigger criteria attempts to ascertain a user's intent.The intent may be determined with a degree of confidence that is builtinto the trigger. Once the threshold degree of confidence is satisfiedthen the trigger is satisfied. The degree of confidence and trigger maybe analyzed using a machine-learning algorithm. The machine-learningalgorithm may be trained using sample data. Once initially trained, themachine-learning algorithm may evaluate the behavior of the specificuser as well as hundreds of thousands of other users that use thecontextual service. The machine-learning algorithm identifies patternsin user behavior that indicate when a user is completing a task and whatinformation will be useful to complete the task. For example, uponpresenting a search interface that is not selected or used by the userin a scenario, the machine-learning algorithm will lower the probabilitythat other users want the search interface when the same contextualcharacteristics occur in the future.

At step 640, content used to provide the contextual service isgenerated. The content is generated by the contextual-service provider.The content may be different depending on the contextual service to beprovided. For example, when the contextual service is autosuggest thenone or more autosuggestions are generated as the content. When thecontextual service is knowledgebase entries, then factual informationrelated to subject matter within a document being composed may beretrieved from a knowledge base and provided as content. In someembodiments, the content may also include formatting used to present thecontent within a contextual-service interface. The content may specify acontextual-service interface for the receiving device to use.

At step 650, the content is communicated to the IME program. The IMEprogram may then use the content to present the contextual service tothe user in conjunction with the active application or task the user isattempting to complete. The contextual service may be provided to theuser in an overlay. In another embodiment, the active program may beadapted to consume the contextual service in a way that is integratedwith the active interface.

Turning now to FIG. 7, a method 700 for providing contextual services inassociation with an input mechanism is provided, in accordance with anembodiment of the present invention. Method 700 may be performed by aninput mechanism program miming on a computing device, such as a laptop,tablet, or mobile device. At step 710, user input is received at an IMEprogram running on a computing device area as mentioned previously, theIME program enables and associated input mechanism. Exemplary inputmechanisms include a hard keyboard, a touch screen keyboard, a gestureinterface, a speech interface, and a writing interface. Gesture inputoccurs when a user motions with their body, or body parts, in a gesturethat is captured in image data by an input device, such as a videocamera, still camera, web camera, depth camera or the like. For example,a user may perform a thumbs-up gesture intending to turn up thetelevision volume. There could be an entire class of gestures includingright-handed gestures, left-handed gestures, occluded gestures, andmulti-person gestures. For sake of example, the volume gesture could beright handed. An input fragment could be the user raising his righthand, without yet observing a raised thumb. In this situation, a volumeinterface could be opened as a contextual service, upon determining theuser has demonstrated, through context and the input fragment, an intentto adjust the volume.

In this case, the IME program may receive the image data and generate acorresponding output with meaning to an application. The gesture couldselect a letter or character or word or provide a control instructionsuch as to navigate to a different page of an interface.

At step 720, the user input is communicated to an active applicationrunning on the device. For example, the user input could be communicatedto a web browser, word processing application, email application, orsome other application running on the computing device. Embodiments ofthe present invention use describing IME program is able to communicatewith multiple applications. As mentioned, the IME program may beassociated with the operating system of the computing device. The userinput may be a keystroke, text fragment, word or phrase. A keyboard ismore likely to generate input comprising a single character whereas agesture or speech interface is more likely to generate words or phrasesas input.

At step 730, the user input is communicated to a remotecontextual-service provider, such as contextual-service provider 240.The user input may be a keystroke, text fragment or a series of words.Embodiments of the present invention may group user input in ameaningful way for communication purposes. In one embodiment, eachkeystroke is communicated individually. In another embodiment, a seriesof keystrokes are communicated together. In yet another embodiment, afilter is in place on the computing device to communicate a thresholdgroup of meaningful keystrokes together that are likely suitable for useby the contextual service. For example, when more than four charactersare entered without a space the four letters may be communicatedtogether as a user input. A threshold could be set at different amountsof letters, but the purpose is to avoid evaluating short words that areunlikely to need composition assistance. The user is more likely toconsume composition assistance with longer words, such as those havingfour or more characters. The above example is from the English language,but different thresholds could be in place for other languages such asAsian languages that use mostly characters without spaces. When userinput is not communicated immediately because of a threshold filter, theinput may be subsequently communicated as part of contextual informationthat is provided.

At step 740, a contextual service instruction is received from theremote contextual service provider. The contextual-service instructioncomprises contextual information needed to provide one or morecontextual services. For example, contextual information needed toprovide composition assistance may be one or more words to suggest tothe user as they are inputting text. For a contextual search, one ormore search results may be provided within the contextual information.The search result could include links to webpages, images, audio files,videos, applications that may help the user perform a task, and thelike. In addition to the contextual information, the contextual-serviceinstructions may define aspects of a contextual interface that is to bepresented. For example, the size and location of the contextualinterface may be presented within the instructions. In anotherembodiment, the IME program determines the characteristics of thecontextual interface without input from the contextual-service provider.

At step 750, a contextual service is output for display. The contextualinterface is output for display in conjunction with a user interfaceassociated with an active application to which the user input wasprovided. For example, the potential interface may be a search interfacepresented off to the side of a word-processing window. The contextualinterface may take the form of an overlay. The size and location of thecontextual interface may be selected to avoid interference with areas ofthe active interface that are a present focal point for the user. Forexample, the user may be typing a sentence at part of a document. Thearea adjacent to and around the sentence may be off limits for certaincontextual interfaces, such as search results. On the other hand,composition assistance may be provided directly below text as it isentered. The user then they may select the suggested word or phraseinstead of continuing to type the entire word. Each auto suggested wordor auto completed word may be associated with a short key that the usermay press to select that particular word or phrase. For example, eachauto suggested word could be associated with a number that the user cantype to select the word.

In one embodiment, the input mechanism includes a control that allowsthe user to turn on or turn off the contextual service functionality.For example, a hard or soft keyboard may include a dedicated key thatactivates and deactivates the contextual service functionality. In oneembodiment, upon activating the service, a search intent isautomatically inferred and contextual search results are generated basedon recent user input in the context.

Turning now to FIG. 8, a method 800 of providing a contextual serviceinterface in association with an input mechanism is provided, inaccordance with an embodiment of the present invention. At step 810, auser input is received at an input mechanism program running on acomputing device. The IME program enables and associated input mechanismthat provides input to multiple applications on a computing device. Atstep 820, the user input is communicated to an active applicationrunning on the device. The active application is the application withwhich the user is presently interacting. As is known, multipleapplications may be running simultaneously on a computing device. And attimes many of the interfaces associated with these applications may beviewed simultaneously. However, the IME program is only inputting textinto one application at a time. The application that is receiving theinput is said to be the active application. The IME program sends theuser input to the active application. For example, the IME program for akeyboard may receive keystrokes that are communicated to the active wordprocessing application.

At step 830, contextual interface is generated it offers to provide oneor more contextual services related to the user input and context in theactive application. The contextual interface may take the form of anoverlay and may be split into multiple overlays. That is to say thecontent of contextual interface may comprise a composition assistantthat is located at a first part of the screen near the text beingimplemented and a second interface that provides search results that areseparate from the composition assistance.

At step 840, a user selection of a contextual service offered in thecontextual interface is received. For example, the user may haveselected composition assistance, such as grammatical or usageassistance. At step 850, the contextual service is initiated. Initiatingthe contextual service may include communicating with acontextual-service provider that is remote from the computing device. Inanother embodiment, initiating the contextual service causes thecontextual interface to be updated to provide a fuller explanation ofavailable contextual available information related to the contextualservice. For example a full entry related to grammatical usage may beprovided where only a snippet was shown.

While the invention is susceptible to various modifications andalternative constructions, certain illustrated embodiments thereof areshown in the drawings and have been described above in detail. It shouldbe understood, however, that there is no intention to limit theinvention to the specific forms disclosed, but on the contrary, theintention is to cover all modifications, alternative constructions, andequivalents falling within the spirit and scope of the invention. Thepresent invention has been described in relation to particularembodiments, which are intended in all respects to be illustrativerather than restrictive.

What is claimed is:
 1. One or more computer storage media havingcomputer-executable instructions embodied thereon, that when executed,cause a computing device to perform a method for providing a contextualservice in association with an input mechanism, the method comprising:receiving over a computer network, at a contextual-service provider, aninput fragment from an input mechanism (“IME”) program running on acomputing device, the input fragment comprising letters that do notpresently constitute a full word; receiving, at the contextual-serviceprovider, information from the IME program that describes acharacteristic of an active application into which the IME program inputthe input fragment; determining, at the contextual-service provider,that the characteristic and the input fragment satisfy a contextualservice's trigger criteria that causes a contextual service to begenerated; generating content used to provide the contextual service;and communicating the content to the IME program.
 2. The one or morecomputer storage media of claim 1, wherein the IME program enables atouchscreen keyboard.
 3. The one or more computer storage media of claim1, wherein the characteristic is relevant to determining what task auser of the computing device is presently completing.
 4. The one or morecomputer storage media of claim 1, wherein the characteristic is textwithin a document presently being composed on the computing device. 5.The one or more computer storage media of claim 4, wherein thecontextual service is an autosuggested word or phrase that does notinclude the input fragment, but that is related by subject matter to aword that that includes the input fragment.
 6. The one or more computerstorage media of claim 1, wherein the contextual service is a group offacts describing an object with a name that includes the input fragment.7. The one or more computer storage media of claim 1, wherein thecontextual service is a search result generated in response to a querycomprising the input fragment.
 8. A method for providing a contextualservice in association with an input mechanism, the method comprising:receiving a user input at an input mechanism (“IME”) program running ona computing device, the IME program enabling an associated inputmechanism; communicating the user input from the IME program to anactive application running on the computing device; communicating theuser input from the IME program to a remote contextual-service provider;communicating from the IME program information that describes acharacteristic of the active application to the remotecontextual-service provider; receiving at the IME program acontextual-service instruction from the remote contextual-serviceprovider, the contextual-service instruction comprising informationneeded to provide one or more contextual services; and outputting fordisplay, a contextual interface that presents a contextual service. 9.The method of claim 8, wherein the IME program enables a gestureinterface that uses image data to identify the user input.
 10. Themethod of claim 8, wherein the input mechanism is a touchscreenkeyboard, and wherein the input is a text fragment forming less than afull word.
 11. The method of claim 8, wherein the method furthercomprises receiving a user interaction with the contextual interface,automatically generating an input for the active application, andcommunicating the input to the active application.
 12. The method ofclaim 11, wherein the active application is a web browser open to a webpage having an input box, the user input is text entered into the inputbox, and the contextual service is a search result generated in responseto the user input.
 13. The method of claim 11, wherein the activeapplication is a web browser open to a web page having an input box, theuser input is text entered into the input box, and the contextualservice is a suggested query generated in response to the user input.14. The method of claim 8, wherein the input mechanism is a keyboard,and wherein the method further comprises receiving an indication that acontextual-service key on the keyboard has been selected and, inresponse, activating contextual service functionality associated withthe IME program.
 15. A computing system comprising: a processor; andcomputer storage memory having computer-executable instructions storedthereon which, when executed by the processor, configure the computingsystem to: receive a user input at an input mechanism (“IME”) programrunning on a computing device, the IME program enabling an associatedinput mechanism that provides input to multiple applications on thecomputing device, the user input comprising letters that do notpresently constitute a full word; communicate the user input from theIME program to an active application running on the computing device;communicating the user input from the IME program to a remotecontextual-service provider; receiving at the IME program acontextual-service instruction comprising information needed to provideone or more contextual services from the remote contextual-serviceprovider; generate, by the IME program, a contextual interface thatoffers to provide one or more contextual services related to the userinput and context in the active application; receive a user selection ofa contextual service offered in the contextual interface; and initiatethe contextual service.
 16. The computing system of claim 15, whereinthe contextual interface is an overlay presented with an interfaceassociated with the active application.
 17. The computing system ofclaim 15, wherein the contextual service is a composition-assistanceservice and the contextual interface is displayed adjacent to a pointwhere the user input is added within an interface associated with theactive application.
 18. The computing system of claim 15, wherein thecontextual service associates a user with multiple devices through alogin to the service and builds a contextual service profile usinginformation from the multiple devices.
 19. The computing system of claim18, wherein the communications include one or more social posts from asocial network and emails addressed to or from an address associatedwith the communication being composed.
 20. The computing system of claim15, wherein the computing system is further configured to communicatethe user selection to a contextual service provider that uses the userselection to determine circumstances when the contextual service isrelevant to a user.